Discrete Optimization for Optical Flow

In this supplementary material we first provide the parameters we used in our discrete optical flow model. Next, we present histograms of the statistics of optical flow for MPI Sintel and KITTI which we use to select the upper bounds for the magnitude of flow vectors in our model. Finally, more detailed qualitative results on the processed datasets are shown, including comparisons to state-of-the-art approaches. 1 Parameters The parameters in our model were set using block coordinate descent on the respective training data. In this section we report the values we used for the experimental evaluation on MPI Sintel. For reproducibility we also report the parameters we used for the feature descriptor. Parameter Symbol Value Relative weight of unary and pairwise terms γ 0.95 Truncation threshold of the data term τφ 2.5 Truncation threshold of the smoothness term τψ 15 Edge weight parameter α 10 Size of the label set L 500 Number of neighbors to be sampled N 200 Stride for discrete CRF in pixels 4 Standard deviation for sampling neighbors 5 Similarity threshold (fw-/bw-check) 10 Minimum segment size (outlier removal) 100 Consistency threshold (outlier removal) 10 Table 1. Parameters of the proposed model. In our implementation we use the relative weight γ to balance the data and smoothness terms E(l) = (1 − γ) ∑ p∈P φp(lp) } {{ } data + γ ∑ p∼q ψp,q(lp, lq) } {{ } smoothness In Equation (1) of the paper we encode this weight as λ in for the sake of simplicity. 2 Moritz Menze, Christian Heipke, and Andreas Geiger DAISY Parameters We leverage the DAISY descriptor [4] due to its computational efficiency and its robustness against changes in illumination. The parameter setting is adapted to our application as stated in Table 2. Note that due to the global optimization performed by our model, a small radius of 5 pixels led to the best results. Parameter Symbol Value Radius R 5 Radius Quantization No. Q 4 Angular Quantization No. T 4 Histogram Quantization No. H 4 Grid Point No. S 1 Descriptor Size DS 68 Table 2. Parameters of the DAISY-Descriptor. 2 Ground Truth Statistics To avoid grossly wrong flow proposals, we limit the admissible flow range to ±250 pixels. As evidenced by the histograms of the respective ground truth flow vectors in Fig. 1, this flow range is appropriate for the datasets considered in this paper. For the sake of simplicity, we employ a single (loose) upper bound which is valid for all scenarios under consideration. u-flow (Px) -200 0 200 F re q u e n c y ( % ) 0 10 20 30 40 50 60 70 -87.3 73.3 v-flow (Px) -100 -50 0 50 100 F re q u e n c y ( % ) 0 10 20 30 40 50 60 70 -50.3 50.6 u-flow (Px) -200 0 200 F re q u e n c y ( % ) 0 10 20 30 40 50 60 70 -114.8 133.5 v-flow (Px) -100 -50 0 50 100 F re q u e n c y ( % ) 0 10 20 30 40 50 60 70 -5.0 49.3 Fig. 1. This figure shows histograms of the optical flow u-component (red) and the optical flow v-component (blue) for the training sets of Sintel [1] (top) and KITTI [2] (bottom). Vertical lines indicate the 0.01 and 0.99 percentile, respectively. Supplementary Material: Discrete Optimization for Optical Flow 3 3 Qualitative Results In this section, we provide additional qualitative results. In addition to our final result, we include visualizations of the individual stages of our algorithm. “Ours Forward” refers to the flow map from block coordinate descent as described in Section 3.3 in the paper and “Ours Clean” comprises our outlier rejection step from Section 3.4. The following figures show comparisons of our post-processed results (“Ours+DeepFlow” and “Ours+EpicFlow”) to the full DeepFlow [5] and EpicFlow [3] pipelines which use DeepMatches [5] as features. At the top of each error map in the right column we specify the percentage of outliers and the endpoint error (EPE). The color coding visualizes outliers (> 3 px EPE) in red and inliers (< 3 px EPE) in blue on a logarithmic color scale. The reference images are the same as shown in the paper, the sixth result of each dataset shows a failure case (Fig. 7 and Fig. 13). 4 Moritz Menze, Christian Heipke, and Andreas Geiger Fig. 2. Qualitiative Results. From top-to-bottom: Reference data (topmost row), optical flow result and error map of the oracle solution, our result without refinement, our result with outlier removal, our method with DeepFlow refinement, the result of DeepFlow [5], our method with EpicFlow refinement, the result of EpicFlow [3]. Supplementary Material: Discrete Optimization for Optical Flow 5 Fig. 3. Qualitiative Results. From top-to-bottom: Reference data (topmost row), optical flow result and error map of the oracle solution, our result without refinement, our result with outlier removal, our method with DeepFlow refinement, the result of DeepFlow [5], our method with EpicFlow refinement, the result of EpicFlow [3]. 6 Moritz Menze, Christian Heipke, and Andreas Geiger Fig. 4. Qualitiative Results. From top-to-bottom: Reference data (topmost row), optical flow result and error map of the oracle solution, our result without refinement, our result with outlier removal, our method with DeepFlow refinement, the result of DeepFlow [5], our method with EpicFlow refinement, the result of EpicFlow [3]. Supplementary Material: Discrete Optimization for Optical Flow 7 Fig. 5. Qualitiative Results. From top-to-bottom: Reference data (topmost row), optical flow result and error map of the oracle solution, our result without refinement, our result with outlier removal, our method with DeepFlow refinement, the result of DeepFlow [5], our method with EpicFlow refinement, the result of EpicFlow [3]. 8 Moritz Menze, Christian Heipke, and Andreas Geiger Fig. 6. Qualitiative Results. From top-to-bottom: Reference data (topmost row), optical flow result and error map of the oracle solution, our result without refinement, our result with outlier removal, our method with DeepFlow refinement, the result of DeepFlow [5], our method with EpicFlow refinement, the result of EpicFlow [3]. Supplementary Material: Discrete Optimization for Optical Flow 9 Fig. 7. Qualitiative Results. From top-to-bottom: Reference data (topmost row), optical flow result and error map of the oracle solution, our result without refinement, our result with outlier removal, our method with DeepFlow refinement, the result of DeepFlow [5], our method with EpicFlow refinement, the result of EpicFlow [3]. 10 Moritz Menze, Christian Heipke, and Andreas Geiger Fig. 8. Qualitiative Results. From top-to-bottom: Reference data (topmost row), optical flow result and error map of the oracle solution, our result without refinement, our result with outlier removal, our method with DeepFlow refinement, the result of DeepFlow [5], our method with EpicFlow refinement, the result of EpicFlow [3]. Supplementary Material: Discrete Optimization for Optical Flow 11 Fig. 9. Qualitiative Results. From top-to-bottom: Reference data (topmost row), optical flow result and error map of the oracle solution, our result without refinement, our result with outlier removal, our method with DeepFlow refinement, the result of DeepFlow [5], our method with EpicFlow refinement, the result of EpicFlow [3]. 12 Moritz Menze, Christian Heipke, and Andreas Geiger Fig. 10. Qualitiative Results. From top-to-bottom: Reference data (topmost row), optical flow result and error map of the oracle solution, our result without refinement, our result with outlier removal, our method with DeepFlow refinement, the result of DeepFlow [5], our method with EpicFlow refinement, the result of EpicFlow [3]. Supplementary Material: Discrete Optimization for Optical Flow 13 Fig. 11. Qualitiative Results. From top-to-bottom: Reference data (topmost row), optical flow result and error map of the oracle solution, our result without refinement, our result with outlier removal, our method with DeepFlow refinement, the result of DeepFlow [5], our method with EpicFlow refinement, the result of EpicFlow [3]. 14 Moritz Menze, Christian Heipke, and Andreas Geiger Fig. 12. Qualitiative Results. From top-to-bottom: Reference data (topmost row), optical flow result and error map of the oracle solution, our result without refinement, our result with outlier removal, our method with DeepFlow refinement, the result of DeepFlow [5], our method with EpicFlow refinement, the result of EpicFlow [3]. Supplementary Material: Discrete Optimization for Optical Flow 15 Fig. 13. Qualitiative Results. From top-to-bottom: Reference data (topmost row), optical flow result and error map of the oracle solution, our result without refinement, our result with outlier removal, our method with DeepFlow refinement, the result of DeepFlow [5], our method with EpicFlow refinement, the result of EpicFlow [3]. 16 Moritz Menze, Christian Heipke, and Andreas Geiger